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This is a request for filing a Continuation application under 37 CFR § 1.53(b) 
of Application Serial No. 08/623,610 which was filed 3/29/96 and is entitled 
" Interactive Entertainment System For Presenting Supplemental Interactive Content 
Together With Continuous Video Programs". The prior application was submitted 
in the names of Shoff et al.. The original declaration showing their address(es) is 
also enclosed. 

The prior application was a complete application as set forth in 
37 CFR §1.5 1(a). The application papers being filed herewith are a true copy of the 
prior application as filed including the specification (with claims), drawings, oath or 
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declaration showing the signature(s), and any amendments referred to in the oath or 
declaration to complete the prior application. No amendments referred to in the oath 
or declaration filed to complete the prior application introduced new matter therein. 
No abandonment or termination of proceeding has occurred in the above-identified 
prior application. 

The filing date of pending application Serial No. 08/623,610 is hereby 
claimed under 35 U.S.C. §120. 

A preliminary amendment is enclosed. 

The Commissioner is hereby authorized to charge fees under 37 CFR §1.16 
and §1.17 associated with this communication, or credit any overpayment to Deposit 
Account No. 50-0463. 

A previous assignment to Microsoft Corporation has been executed by the 
inventors in the prior application, and is applicable to the present application. A 
copy of the assignment is enclosed herewith. 

A previous power of attorney in the prior application is given to Lewis C. 
Lee, Reg. No. 34,656, Daniel L. Hayes, Reg. No. 34,618, Katie E. Sako, Reg. No. 
32,628 and Daniel D. Crouse, Reg. No. 32,022. Please send correspondence to: LEE 
& HAYES, PLLC, 421 W. Riverside Avenue, Suite 500, Spokane, 
Washington, 99201. Direct telephone calls to: Lewis C. Lee (509) 324-9256. 
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Respectfully Submitted, 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Priority Application Serial No 08/623,610 

Priority Filing Date March 29, 1996 

Inventor Shoffetal. 

Priority Group Art Unit 271 1 

Priority Examiner R. House 

Attorney's Docket No MS1-089USC1 

Title: Interactive Entertainment System for Presenting Supplemental Interactive 
Content Together with Continuous Video Programs 

PRELIMINARY AMENDMENT 

To: Commissioner of Patents and Trademarks, 
Washington, D.C. 20231 

From: Lewis C. Lee (Tel. 509-324-9256; Fax 509-928-2642) 
Lee & Hayes, PLLC 
421 W. Riverside Avenue, Suite 430 
Spokane, WA 99201 

In the Claims : 

Claims 1-55 are contained in the Application as filed. 

Kindly cancel claims 1-55. 

Please add the following new claims 56-74. 

—56. A viewer computing unit for receiving and displaying continuous 
video content programs, comprising: 
a memory; 

a processor programmed to determine whether the video content programs 
are interactive; 

a tuner to tune to channels carrying the video content programs; and 



video content programs, comprising: 
a memory; 

a processor programmed to d 
are interactive; 
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an Internet browser stored in the memory, the Internet browser being 
dynamically loadable for execution on the processor when the tuner is tuned to a 
channel carrying a video content program that is interactive.— 

—57. A viewer computing unit as recited in claim 56, and further 
comprising: 

an electronic programming guide (EPG) stored in the memory and 
executable on the processor to organize programming information, the EPG 
associating a target specification to a target resource with a video content program; 
and 

the Internet browser activating the target resource when the tuner is tuned to 
the video content program. — 

-58. A method for presenting an interactive program, comprising the 
following steps: 

receiving a program as a continuous stream of video data; 

receiving digital data for supporting interactive functionality in relation to 
the program; 

displaying the program within a program boundary on a visual display 

screen; 

presenting supplemental content from the digital data in a presentation 
format on the visual display screen which enables the interactive functionality; 

dynamically controlling location and shape of the program boundary and 
the presentation format of the supplemental content relative to the program 
boundary on the visual display screen; and 
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presenting the supplemental content outside of the program boundary.— 



—59. A method for presenting an interactive program, comprising the 
following steps: 

receiving a program from a first source as a continuous stream of video 

data; 

receiving digital data from a second source that is different than the first 
source for supporting interactive functionality in relation to the program; 

displaying the program within a program boundary on a visual display 

screen; 

presenting supplemental content from the digital data in a presentation 
format on the visual display screen which enables the interactive functionality; and 

synchronizing presentation of the supplemental content to corresponding 
points in the program.-- 

—60. A computer programmed to perform the following steps: 

receiving a program from a first source as a continuous stream of video 

data; 

receiving digital data from a second source that is different than the first 
source for supporting interactive functionality in relation to the program; 

displaying the program within a program boundary on a visual display 

screen; 

presenting supplemental content from the digital data in a presentation 
format on the visual display screen which enables the interactive functionality; and 
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dynamically controlling location and shape of the program boundary and 
the presentation format of the supplemental content relative to the program 
boundary on the visual display screen.-- 

—61. A computer-implemented method for activating interactive 
supplemental content for a video content program upon tuning to a channel 
carrying the program, comprising the following steps: 

determining if the program is interactive compatible, where interactive 
compatible programs are associated with target resources containing data which 
support interactive functionality in conjunction with the associated programs, the 
target resources being located by corresponding target specifications; and 

in an event that the program is interactive compatible, retrieving a target 
specification associated with the program and dynamically launching an Internet 
browser to activate the target resource in support of interactive functionality for the 
associated program.— 

—62. A computer-implemented method as recited in claim 61, wherein the 
target specifications are correlated with associated programs in a program listing, 
and further comprising the following steps: 

checking the program listing to ascertain whether the program is interactive 
compatible; and 

determining that the program is interactive compatible by presence of a 
target specification being associated with the program in the program listing.— 

—63. A computer programmed to perform the steps recited in claim 61. 
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— 64. A computer-implemented method for activating interactive 
supplemental content for a video content program upon tuning to a channel 
carrying the program, comprising the following steps: 

determining if the program is interactive compatible by checking a channel 
separate from said program channel for presence of the supplemental content in 
conjunction with the program being received on said program channel, where 
interactive compatible programs are associated with target resources containing 
data which support interactive functionality in conjunction with the associated 
programs, the target resources being located by corresponding target 
specifications; and 

in an event that the program is interactive compatible, retrieving a target 
specification associated with the program and dynamically launching an Internet 
browser to activate the target resource in support of interactive functionality for the 
associated program.- 

—65. A computer-implemented method for activating interactive 
supplemental content for a video content program upon tuning to a channel 
carrying the program, comprising the following steps: 

determining if the program is interactive compatible, where interactive 
compatible programs are associated with target resources containing data which 
support interactive functionality in conjunction with the associated programs, the 
target resources being located by corresponding target specifications; 

displaying an icon to visually inform the viewer that the program is 
interactive compatible; and 
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in an event that the program is interactive compatible, retrieving a target 
specification associated with the program and launching an Internet browser to 
activate the target resource in support of interactive functionality for the associated 
program.— 

—66. A computer-implemented method for activating interactive 
supplemental content for a video content program upon tuning to a channel 
carrying the program, comprising the following steps: 

determining if the program is interactive compatible, where interactive 
compatible programs are associated with target resources containing data which 
support interactive functionality in conjunction with the associated programs, the 
target resources being located by corresponding target specifications; 

displaying the interactive supplement content in response to the viewer 
activating the icon; and 

in an event that the program is interactive compatible, retrieving a target 
specification associated with the program and launching an Internet browser to 
activate the target resource in support of interactive functionality for the associated 
program. - 

—67. A computer-implemented method for activating interactive 
supplemental content for a video content program upon tuning to a channel 
carrying the program, comprising the following steps: 

determining if the program is interactive compatible, where interactive 
compatible programs are associated with target resources containing data which 
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support interactive functionality in conjunction with the associated programs, the 
target resources being located by corresponding target specifications; 

in an event that the program is interactive compatible, retrieving a target 
specification associated with the program and launching an Internet browser to 
activate the target resource in support of interactive functionality for the associated 
program; and 

automatically displaying the interactive supplement content together with 
the interactive compatible program- 

—68. A method for creating a data structure in a storage medium that is used 
to organize programming information, comprising the following steps: 

forming data fields in a storage medium to hold programming information 
pertaining to video content programs, some of the data fields holding text-based 
data; and 

adding a target specification which references a target resource containing 
data that supports interactive functionality with respect to various ones of the video 
content programs by at least one of (1) forming a separate data field to hold the 
target specification for an associated video content program, or (2) embedding the 
target specification within the text-based data held in a data field.— 

—69. A computer programmed to perform the steps recited in claim 68.— 

—70. A storage medium having a data structure created according to the 
steps recited in claim 68.— 
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—71. A method for authoring an interactive entertainment program, 
comprising the following steps: 

constructing digital data to support interactive functionality with a video 
content program, the digital data being configured to permit a viewer to 
interactively control display of supplemental content along with the video content 
program; 

defining a display layout of how the supplemental content and the video 
content program are displayed in relation to one another; 

developing timing information to synchronize presentation of the 
supplemental content in conjunction with the video content program; and 

encoding the digital data with instructions to dynamically change the 
display layout of the supplemental content and the video content program and to 
alter the display layout of the supplemental content and the video content program 
in response to the timing information. — 

—72. A method for authoring an interactive entertainment program, 
comprising the following steps: 

constructing digital data to support interactive functionality with a video 
content program, the digital data being configured to permit a viewer to 
interactively control display of supplemental content along with the video content 
program; 

defining a display layout of how the supplemental content and the video 
content program are displayed in relation to one another ; 

encoding the digital data with instructions to dynamically change the 
display layout of the supplemental content and the video content program; and 
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storing the digital data with instructions as a target resource in a storage 
medium. - 

—73. A target resource stored in a storage medium which is constructed 
according to the steps recited in claim 39.-- 

—74. A computer programmed to perform the following steps: 
constructing digital data to support interactive functionality with a video 
content program, the digital data being configured to permit a viewer to 
interactively control display of supplemental content along with the video content 
program; 

defining a display layout of how the supplemental content and the video 
content program are displayed in relation to one another; and 

encoding the digital data with instructions to dynamically change the 
display layout of the supplemental content and the video content program.— 
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REMARKS 

Applicant respectfully requests entry of the above amendments prior to 
examination of the application. Claims 56-74 are pending in this application. 

Claims 56-57 are modified versions of original claims 3-4, as they were 
pending in parent application. 

Claims 58-62 are modified versions of original claims 22-26, as they were 
pending in parent application. 

Claim 63 is a modified version of original claims 32, as it was pending in 
parent application. 

Claims 64-67 are modified versions of original claims 27-30, as they were 
pending in parent application. 

Claim 68-70 are modified versions of original claims 33-35, as they were 
pending in parent application. 

Claim 71-74 are modified versions of original claims 38-41, as they were 
pending in parent application. 

Section 131 Declaration 

A Declaration under 37 C.F.R. §1.131 is submitted herewith to remove the 
Hidary et al. patent as prior art. This patent was cited in the parent application. 
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Conclusion 

All pending claims 56-74 are in condition for allowance. Applicant 
respectfully requests prompt issuance of the subject application. 



Respectfully Submitted, 



Date: XA^im By: 

Lewis C. Lee 
Reg. No. 34,656 
(509)324-9256 ext. 11 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

APPLICATION FOR LETTERS PATENT 



Interactive Entertainment System For Presenting 
Supplemental Interactive Content Together With 
Continuous Video Programs 



Inventor(s): 
Dan Shoff 
Valerie Bronson 
Joseph Matthews, III 
Frank Lawler 



ATTORNEY'S DOCKET NO. MS1-089US 



TECHNICAL FIELD 

This invention relates to interactive entertainment systems, such as 
interactive television or interactive computing network systems.^ More 
particularly, this invention relates to methods for presenting supplemental 
interactive content in conjunction with continuous video programs to enable 
viewer interactivity with the otherwise non-interactive video programs. 

BACKGROUND OF THE INVENTION 

Video content programs, such as TV shows and pay-per-view movies, are 
delivered to a viewer as a continuous data stream. Today, programs are most 
commonly distributed using a wireless broadcast system, or a cable system. In the 
first instance, the programs are broadcast over a wireless network and received at 
individual homes through an antenna or satellite dish. In the latter case, the 
programs are transmitted over cable to set-top boxes resident in the viewers 5 
homes. 

In broadcast distribution systems, there is no opportunity for interactive 
control of the content by the viewer. The viewer simply has an option to watch the 
program, change to a different program, or turn off the television. However, as 
consumers have learned from playing video games on their televisions, non- 
interactive viewing is not nearly as fun or sensory rich as interactive entertainment. 

To enhance the traditional way of viewing television, there has been some 
effort toward the production of interactive programming content. As presently 
contemplated, additional interactive content is created to enhance the existing 
traditional program. This supplemental content is played along with the 
continuous video stream to enable viewers to interact with the program in a more 
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involved manner than simply watching it. The supplemental content might, for 
example, ask the viewer questions about the episode, or play games with the 
viewer that relate to the show, or describe behind-the-scenes aspects of making the 
program. However, there remains a significant hurdle concerning how to create 
and distribute interactive programs. 

Apart from the TV environment, however, computer users are familiar with 
interactive content on their computers. Many users own interactive multimedia 
CD-ROM applications that combine video, audio, pictures, text, and other content 
into a rich and responsive presentation. Familiar examples of interactive computer 
applications include games (e.g., Myst from Broderbund), educational programs 
(e.g., Reader Rabbit series from the Learning Company or Magic Schoolbus series 
from Microsoft), and home entertainment (e.g., Encarta from Microsoft). 

Interactive content is also available from online services over a public 
network. Most notably, the Internet is emerging as a means for supplying video, 
sound, pictures, text, and other multimedia rich resources to a user's computer. 
Through the Internet, users can access a wide variety of resources that are 
maintained on computers located around the world. 

Resources available on the Internet are most commonly presented as 
hypertext. "Hypertext," also referred to as "hypermedia," is a metaphor for 
presenting information in which text, images, sounds, and actions become linked 
together in a complex, non-sequential web of associations that permit a user to 
browse through related topics, regardless of the presented order of the topics. 
Hypermedia content is widely used for navigation and information dissemination 
on the "World-Wide Web" (WWW or Web) of the Internet. An application 
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program referred to as a "Web browser" is normally used to retrieve and render 
hypermedia content from the WWW. 

Despite the development of interactive entertainment in the computer 
sector, there has been little activity spilling over into the traditional television 
world. The TV environment poses a problem in that the programs are typically 
delivered as a broadcast of a continuous data stream, which inherently does not 
support interactive control. Interlacing interactive content presents a difficult 
design issue. 

One proposed solution is a technology referred to as "Intercast technology" 
which is available from the Intercast Industry Group, a consortium of leading 
television networks, program hardware vendors, and software vendors. The 
Intercast technology presents both television programs and Internet data together 
on the same television or computer monitor, but with separate and predefined 
panes. 

Fig. 1 shows a screen 14 (television or computer monitor) which illustrates 
the Intercast technology. The screen 14 is divided into panes, as represented by 
panes 16-18. Pane 16 contains the television program resulting from the video 
data. Pane 17 contains a hypermedia document, such as a Web page, that is 
provided by the Internet data. A third pane 18 can be used to show additional data, 
such as advertisements or the like. 

The Internet data is combined with the video data of the television program 
to form a single signal that is broadcast to the viewer. The Internet data is 
transmitted during the vertical blank interval (VBI) between successive frames of 
the video data. The Internet data and video data are separated at the viewer's 
computer and presented simultaneously within their respective panes. 
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The drawback with the Intercast technology is that it rigidly adheres to the 
paned presentation. The television pane 16 is a self-contained pane which is 
dedicated to showing only the video program, and the Web pane 17 is a separately 
self-contained pane which is reserved exclusively for Web content. Content 
providers who develop the Internet data have no control over how the television 
program and interactive supplemental content is presented to the viewer. They can 
simply control how the data is presented within its own box 17. Accordingly, the 
content providers are significantly limited in what they can create in the way of a 
full interactive media event. 

The inventors have developed a better way of creating and distributing 
interactive programming that frees the content providers of these restrictions. 

SUMMARY OF THE INVENTION 

This invention concerns an interactive entertainment system for supplying 
interactive supplemental content along with continuous video content programs to 
viewers. The programs are supplied from a program provider, such as a cable 
headend or a broadcast station, over a distribution network to a viewer computing 
unit. The programs are continuous, non-interactive data streams, such as television 
shows, movies, or other video content. 

The viewer computing unit is implemented as a television, a television/set- 
top box unit, a personal computer, or the like. It has a processor and memory. An 
electronic programming guide (EPG) is stored in the memory and executable on 
the processor to organize programming information that is descriptive of the video 
content programs. The EPG maintains a data field that indicates whether the video 
content program is interactive. The EPG data field contains a pointer, universal 
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resource locator, or other target specification to the target resource that supports 
the interactive content and correlates this data field with the particular program 
that the supplemental content is designed to enhance. Accordingly, if a target 
resource is listed in conjunction with a particular program, the program is 
recognized as being interactive. The supplemental content can be developed and 
provided by the same provider that distributes the video content program, or by an 
independent service provider. 

When the viewer tunes to a particular channel, the viewer computing unit 
consults the EPG to determine if the present program is interactive. If it is, the 
viewer computing unit launches an interactive support module, such as an Internet 
browser. This browser is kept in memory and is dynamically loadable for 
execution on the processor when the viewer tunes to a channel carrying a video 
content program that the EPG identifies as interactive. The viewer computing unit 
also depicts a small icon or other indicia to alert the viewer that the program is 
interactive. The viewer can click on or otherwise activate the icon to enter the 
interactive mode and display the supplemental content. As an alternative, the 
supplemental content can be automatically displayed in response to launching the 
Internet browser. 

The Internet browser uses the target specification in the EPG to start the 
target resource. The target resource contains the supplemental content to enhance 
the television program. The supplemental content might be, for example, 
questions about the program, games, trivia information, facts about the actors and 
producers, information on other episodes, advertisements, a listing of products or 
memorabilia about the program, and so on. The supplemental content and 
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program can be transmitted together as a single signal, which is separated at the 
viewer computing unit, or separately over two channels or two different networks. 

The target resource also contains display layout instructions prescribing 
how the supplemental content and the video content program are to appear in 
relation to one another when displayed on the television or monitor. When the 
data from the target resource is downloaded to the viewer computing unit, the 
processor is responsive to the layout instructions obtained from the target resource 
to display the supplemental content concurrently with the video content program. 

Embedding the layout instructions in the supplemental content 
advantageously returns control of the presentation to the content developers. The 
developers can now arrange the data and video in any manner they choose. The 
developers are not restricted to specific fixed panes, as in the prior art. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is an illustration of a screen showing a paned presentation format for 
presenting television programs and Internet data according to prior art Intercast 
technology. 

Fig. 2 is a diagrammatic illustration of an interactive entertainment system 
according to one implementation. The Fig. 2 implementation is representative of 
an interactive TV system. 

Fig. 3 is a simplified example of data fields in a data structure maintained 
by an electronic programming guide. 

Fig. 4 is a diagrammatic illustration of an interactive entertainment system 
according to a second implementation, which is representative of an interactive 
integrated PC-TV system. 
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Fig. 5 is a block diagram of a viewer computing unit according to an aspect 
of this invention. 

Figs. 6 and 7 show a flow chart for operating an interactive entertainment 
system. 

Fig. 8a-8c are a series of exemplary screen illustrations showing different 
display layouts of the video program and the supplemental content. 

Fig. 9 is a flow chart of a method for authoring an interactive entertainment 
program. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Fig. 2 shows an interactive entertainment system 20 implemented as an 
interactive television (ITV) system. System 20 includes a centralized headend 22 
which is configured to provide continuous video content programs to multiple 
subscribers. These programs might include traditional broadcast TV shows, 
movies, games, and the like. The programs are transmitted as a continuous data 
stream, which is non-interactive in itself. 

Each subscriber has a viewer computing unit 24. In the illustrated 
implementation, the viewer computing unit is embodied as a set-top box (STB) 26 
coupled to a television (TV) 28. The STB 26 receives digital video signals from 
headend 22 and controls which programs are displayed on the TV 28. The STB 26 
is controlled by the viewer through use of a panel control located on its face, or a 
hand held remote control unit 30. Instead of a separate STB, however, a viewer 
computing unit can be incorporated in the TV itself. In addition to televisions, the 
viewer computing unit might be implemented with other types of visual display 
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units, such as a computer monitor, as is described below with respect to the Fig. 4 
implementation. 

Each viewer computing unit 24 is configured to run an electronicj)rogram 
guide (EPG). The EPG provides an on-screen listing of various programs or 
program titles. The listing is organized in a predetermined arrangement that is 
displayed on the television. The EPG might also include other program 
descriptive information, including whether the program is provided in closed 
caption or stereo. 

Headend 22 is interconnected to the subscribers via a distribution structure 
or network 32. The distribution structure 32 can be implemented in different 
ways. One implementation is a multi-tier network which includes a high-speed, 
high-bandwidth fiber optic cable network between the headend and regional 
distribution nodes (not shown), and conventional home entry lines, such as 
twisted-pair lines or coaxial cable, between the distribution nodes and viewer 
computing units. Other implementations include satellite communications (e.g., 
DSS technologies), RF communication, or other wireless technologies. Moreover, 
the network can be constructed using a combination of wireless and wire-based 
technologies. 

In the illustrated implementation, the network 32 provides both the pipeline 
for distribution of programs from the headend to the subscriber, as well as a back 
channel which enables communication from the subscriber to the headend. The 
back channel permits the viewer to send requests and instructions to the headend to 
facilitate interactive control. 

Headend 22 provides video content programs to the viewer computing unit 
24. The programs are embodied as video data streams that are transmitted from 
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headend 22 over distribution structure 32 to the viewer computing unit. The 
headend maintains a database of programs 40 that can be served by a continuous 
media server 42 to individual viewers in an on-demand mode. The headend can 
also retransmit to its subscribers broadcast video signals that it receives from 
another source, such as a satellite feed or another cable system. The video data 
streams are sent to the viewer computing units in digital or analog format. 

An EPG server 44 is provided at headend 22 to serve the programming 
information needed by the EPG at the viewer computing unit 24. The EPG server 
44 is implemented as a structured query language (SQL) database 46 with records 
containing information relating to available shows or programs. 

The headend 22 further includes an enhanced content server 52 which 
serves supplemental interactive content to the viewer computing units to enhance 
or supplement the continuous video streams served by the continuous media server 
42. The supplemental content is stored digitally in database 54 and can be text, 
graphics, video, picture, sound, or other multimedia types. Examples of 
supplemental content include trivia questions or games related to the program, 
advertisements, merchandise or other memorabilia, hyperlinks to similar programs 
of similar type or starring the same actor/actress, and so on. 

In the implementation described herein, the supplemental content is 
constructed as a hypertext file which is rendered by a browser. Hypertext, or 
hypermedia, is a metaphor for presenting information in which text, images, 
sounds, and actions become linked together in a complex, non-sequential web of 
associations that permit a user to browse through related topics, regardless of the 
presented order of the topics. Hypermedia content is widely used for navigation 
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and information dissemination on the "World-Wide Web" (WWW or Web) of the 
Internet. 

Hypermedia content is commonly organized as documents with embedded 
control information. The embedded control information includes formatting 
specifications, indicating how a document is to be rendered by the browser. In 
addition, such control information can include links or "hyperlinks/' which are 
symbols or instructions telling the browser where to find other related documents 
on the Internet. A hyperlink includes a label, which is typically rendered as a 
graphical icon or as highlighted keywords, and an underlying target specification. 
The target specification is set forth in the underlying hypermedia document, but is 
normally invisible to the user. The target specification unambiguously identifies a 
targeted document or resource, specifying the name of the computer on which the 
document resides and the complete file name of the document. 

In concept, the target resource can be virtually any type of object — 
including executable programs, text or multimedia documents, sound clips, audio 
segments, still images, computers, directories, and other hyperlinks. In WWW 
documents, hyperlink targets are most often files that can reside on any computers 
connected to the Internet. However, a hyperlink target can also be a particular 
location within a document, including the document that is currently being 
rendered. Hypertext usage is not limited to the Internet. Various multimedia 
applications utilize hypertext to allow users to navigate through different pieces of 
information content. 

Fig. 3 shows an example data structure 48 which is used by the EPG 
database 46 to organize programming information and to correlate target 
specifications with the programs. The data structure includes a number of data 
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records comprising various data fields 50 for holding programming information. 
The data fields contain program titles, actor names, whether the program has 
closed captioning or stereo audio, the scheduled time of the program, the network 
name, and description text. Each program record in the data structure 48 also 
holds pointers to storage locations within the continuous media server 42 which 
hold the video data streams of the programs corresponding to the program records. 

The data structure 48 includes a data field 58 for holding target 
specifications which reference target resources supporting the supplemental 
content. The data structure correlates the target specifications with the programs 
by associating them within the same program record. The presence of a target 
specification within the data field 58 indicates that the associated program is 
interactive and that complementary content can be displayed in addition to the 
program itself. The target specification can be implemented in different forms. 
The target specification to the supplemental content stored at the headend which 
enhances the "Murder, She Wrote" program is in the form of a pointer 
"CONTENT1." The presence of this pointer in field 58 indicates that the program 
"Murder, She Wrote" is interactive compatible and that the supplemental content 
resides at a memory location identified by the pointer "CONTENT 1 " Similarly, 
the programs "Seinfeld" and "Star Trek" are interactive compatible and have 
associated supplemental content which are located by the hyperlink URLs in 
corresponding data field 58. These URLs reference resources provided by 
independent service providers (described below with reference to Fig. 4). 

A URL (universal resource locator) is a type of target specification used in 
WWW documents to describe everything about a particular resource that a 
browser needs to know to request and render it. The URL describes the protocol a 



Lee & Hayes, PLLC 



1 1 MS$ 39279 032996Ni4C.\IJJSharu(MiSI\089ui\MS/.089USpat,app.cloc 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



browser should use to retrieve the resource, the name of the computer it is on, and 
the path and file name of the resource. 

The following is an example of a URL listed in the EPG data structure 48 in 
association with the "Seinfeld" program: 

http : / /www. nbc . com/ seinf eld 

The "http://" portion of the URL describes the protocol. The letters "http" 
stand for HyperText Transfer Protocol, the set of rules that a browser will follow 
to request a document and the remote server will follow to supply the document. 
The "www.nbc.com" portion of the URL is the name of the remote host computer 
which maintains the document. The last portion "/seinfeld" is the path and file 
name of the document on the remote host computer. 

The development of the EPG data structure 48 includes designating separate 
data fields 50 within a storage medium for holding programming information, and 
dedicating at least one of the data fields 58 to storing target specifications to 
supplemental content. The target specifications can be in the form of memory 
pointers, hyperlinks, URLs, or any other designation for referencing a location 
containing supplemental content. For programs with no corresponding 
supplemental content, the target specifications data field 58 is left empty. 
However, for those programs with supplemental content, the target specification to 
the target resource containing the supplemental content is entered in the data field 
58 and correlated with the program. The correlation is provided through the data 
structure by the placing the target specification within the same data record as the 
program title and other information for a particular program. It is further noted 
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that the target resources can be embedded within other text-based data held in 
other data fields 50 which also relate to the corresponding programs. 

The same or a subset of data structure 48 is employed at the EPG 
application running at the viewer computing unit in the home. The data records 
stored at the headend on the EPG server are transmitted periodically in batch, or 
individually, and cached at the local EPG. The local EPG is thus able to identify 
whether a particular program is interactive compatible by quick reference to the 
locally cached EPG data structure. 

Fig. 4 shows an interactive entertainment system 60 according to a second 
implementation which accommodates third party independent service providers. 
Interactive entertainment system 60 includes headend 22 which supplies programs 
and supplemental content to a viewer computing unit 62. In this implementation, 
the viewer computing unit 62 is implemented as a computer 64 with a monitor 66, 
a central processing unit 68, a keyboard 70, and a mouse 72 (although other input 
devices can be used in addition to, or instead of, the keyboard and mouse). 

The headend 22 serves the programs over a first network 74, which may be 
implemented like the fiber optic distribution structure described above, or as a 
satellite system or other wireless broadcast system, or as a conventional data 
network. It is noted that other program providers may be used instead of a 
headend, such as a broadcast station or an online service provider. 

The interactive entertainment system 60 also includes multiple independent 
service providers (ISPs), as represented by ISP 80, which distribute digital content 
to the viewer computing unit 62 over a second network 82. An example of the 
second network 82 is a public network, such as the Internet. The ISP 80 has a host 
84 and a content database 86 to serve various multimedia content to the viewer's 
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computer. The ISP host 84 stores one or more target resources that can be 
rendered by the viewer computing unit 62. 

The supplemental content provided by the ISP 80 is correlated with the 
programs by data structure 48 as described above. As shown in Fig. 3, the data 
field 58 includes target specifications to supplemental content provided by servers 
other than the program provider which distributes the show. In the illustrated 
example, the programs Seinfeld and Star Trek have associated ISP-served content 
which are located by the hyperlink URLs in corresponding data field 58. It is 
further noted that a program might have more than one source of supplemental 
content. The program record for "Star Trek" has two associated target 
specifications, one that references a target resource at the host computer 
"www.fox.com" and one that references a target resource at the host computer 
"www.collections.com." 

According to this Fig. 4 arrangement, the viewer computing unit 62 
receives a non-interactive, continuous video stream from the headend 22, and 
supplemental interactive content from either the headend 22 or from an 
independent service provider 80. The supplemental content might therefore be 
carried to the viewer's computer over the same channel as the program, over a 
separate channel, or over a separate connection unrelated to the program channels. 
The back channel for facilitating interactive control is provided through either 
network 72 or 82. 

In another implementation, the interactive content can be supplied locally 
by a storage medium, such as a CD-ROM. Suppose, for example, a content 
developer creates an interactive CD-ROM that can be played along with a 
particular movie. The supplemental content is accessed via disk reads to the local 
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storage drive, rather than using URLs to target resources over a network. The 
supplemental content is synchronized with the program using open loop control, 
such as a start time followed by measurable ticks or by frame count. 

Fig. 5 shows a viewer computing unit, referenced generally as numeral 90, 
in more detail. The viewer computing unit 90 includes a processor 92, a volatile 
memory 94, and a program memory 96. The viewer computing unit 90 also has at 
least one receiver, and possibly two receivers, for receiving the video stream from 
the headend and the digital supplemental data from the headend or ISP. The first 
receiver is in the form of a tuner 98 which tunes to the channel or broadcast 
frequency to receive a video data stream from a program source, such as the 
headend, broadcaster, or other program provider. The second receiver, referenced 
generally as 100, can be implemented as a second tuner for receiving the content 
over a cable or wireless distribution network, or a modem for receiving the 
supplemental content over the Internet or other data network. 

The viewer computer runs an operating system 101 which supports multiple 
applications. The operating system 101 is stored in memory and executes on the 
processor. The operating system is preferably a multitasking operating system 
which allows simultaneous execution of multiple applications. The operating 
system 101 employs a graphical user interface windowing environment which 
presents the applications or documents in specially delineated areas of the display 
screen called "windows." One preferred operating system is a Windows® brand 
operating system sold by Microsoft Corporation, such as Windows® 95 or 
Windows® NT or other derivative versions of Windows®. However, other 
operating systems which provide windowing environments may be employed, such 
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as the Macintosh operating system from Apple Corporation and the OS/2 operating 
system from IBM. 

A channel navigator application 102 is stored in program memory 96 and 
executes on the processor 92 to control the tuners 98 and 100 to select a desired 
channel for receiving the video content programs. An EPG application 104 is 
stored in program memory 96 and executes on the processor 92 to organize 
programming information downloaded from the EPG server at the headend. The 
EPG 104 supports a displayable user interface (UI) which visually correlates 
programs titles to scheduled viewing times and tuning information, such as a 
channel, in a scrollable grid format. The viewer computing unit 90 also has an 
interactive support module in the form of browser 106 which is kept in memory 
96. The browser 106 is dynamically loaded on processor 92 when needed to 
render content, such as a hypertext document, from an ISP or other content 
provider. The browser can be implemented as a hyperlink browser, or more 
particularly, as an Internet Web browser. 

Although not shown, the viewer computer might also include on or more 
storage drives, such as a CD-ROM drive, to play interactive CD-ROMs in the 
event that supplemental data is supplied locally on disk. 

Figs. 6 and 7 show a method for operating an interactive entertainment 
system to enhance a conventional continuous video data stream, such as a 
television show, with interactive supplemental content. This method is described 
with additional reference to the screen displays shown in Figs. 8a-8c. 

The method begins when a viewer tunes to a particular channel (step 150 in 
Fig. 6). The channel navigator 102 controls the tuner 98 to tune to the channel. 
The viewer computing unit checks the appropriate channel and time slot of the 
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EPG data structure 48 to determine if the program being carried on the selected 
channel at this time is interactive (step 152). As described above, the presence of a 
target specification in the EPG data field 58 in association with the program is in 
indication that the program is interactive compatible and that there is supplemental 
content for the program. If the data field is empty, indicating that no supplemental 
content exists (i.e., the "no" branch from step 154), the viewer computing unit 
simply displays the video data stream being received through the tuner 98 (step 
156). 

A run-time technique can alternatively be used for detecting whether a 
program is interactive compatible. Rather than checking the EPG data field, the 
viewer computing unit checks a dedicated channel for the existence of new 
supplemental content data. The dedicated channel is separate from the selected 
channel carrying the program so that the supplemental content is received by the 
viewer computing unit currently with the program video data. The existence of a 
supplemental content data stream over the dedicated channel indicates that the 
program being received on the selected channel is interactive compatible. This 
technique can be carried out without reference to the EPG listing. 

If the program is interactive compatible (i.e., the "yes" branch from step 
154), the viewer computing unit retrieves the target specification from the EPG 
data structure (step 158 in Fig. 6). The target specification might be a pointer to a 
memory location at the headend, or a pointer to a memory location on a locally 
running CD-ROM, or a hyperlink to a target resource located at an independent 
service provider. The hyperlink browser 106 is loaded onto the processor to render 
the target resource referenced by the target specification (step 160). 
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At this point, there are several ways to initiate viewing the supplemental 
content. One approach is to permit the viewer to selectively activate the 
interactive mode (i.e., the "viewer activate" branch from step 160). The viewer 
may know that the program is interactive compatible by checking a newspaper 
listing or other program guide. Another way is to have the viewer computing unit 
display an icon or other indicia on the screen to visually inform the viewer that the 
program is interactive compatible (step 162 in Fig. 6). The icon can be overlaid on 
the playing video program in an non-conspicuous manner, like the closed caption 
or stereo labels. 

Fig. 8a shows an example screen 200 having a video program 202 playing 
at full-size on the screen. An icon 204 is displayed at the lower right corner of the 
screen to inform the viewer that the program 202 is interactive compatible. The 
icon 204 can be displayed throughout the program, or faded out after a set time 
period. This initial screen represents a first mode of interactivity, where the 
viewer is presented with the choice of interactive entertainment or non-interactive 
viewing of the program. As long as the viewer does not activate the icon 204, the 
viewer computing unit continues to receive the video content program over the 
selected channel and display that program alone, without any supplemental content 
(steps 166 and 168 in Fig. 6). 

If the viewer decides to enter into an interactive mode, the viewer employs 
a remote control handset, mouse, keyboard, or other mechanism to actuate the icon 
204. This causes the browser 106 to start the target resource located by the target 
specification listed in the EPG data structure (step 170 via the "yes" branch from 
step 164). 
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This leads to another approach to invoking the supplemental content. 
Rather than displaying an icon and waiting for input from the viewer, the viewer 
computing unit can automatically activate the target resource as soon as the 
browser is loaded on the processor (step 170 from the "automatic" branch from 
step 160). 

The target resource contains digital data which supports interactive 
functionality in conjunction with the associated video content program. The 
digital data defines the supplemental content to enable viewer interactivity with the 
video content program. The digital data also defines a display layout prescribing 
how the supplemental content and the video program are to appear in relation to 
one another when displayed on the screen. 

The digital data further defines timing information to synchronize 
presentation of the supplemental content with the video content program. This 
timing information can be implemented in many different ways. One approach is 
to coordinate the supplemental content to a particular program start time, and then 
carefully measure time from that start time to synchronize presentation of the 
supplemental content with the appropriate points in the program. Another 
approach is to characterize the program in terms of frames, and key the 
introduction of supplemental content to the frame numbers. 

The video stream and supplemental content can be transmitted together, as a 
single signal, or separately. In the former implementation, the digital data is 
packaged with the video stream and transmitted as one signal from the headend. 
The viewer computing unit receives the single signal at tuner 98 and separates the 
digital data from the video signal (steps 172 and 174). In the latter 
implementation, the video data stream is received over the selected channel via 
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tuner 98 from the program provider (step 172). The digital data is received from 
the target resource that is activated by the browser (step 174). This target resource 
might be located at the headend, in which case, the data is received over a^separate 
channel using tuner 100; or the target resource might be served by an independent 
service provider, in which case, the data is received over a public network using 
the modem 100. 

At the viewer computing unit, the digital data is deconstructed to extract the 
timing information and the display layout from the supplemental content (step 176 
in Fig. 6 and step 178 in Fig. 7). The display layout defines a program boundary 
within which the program is displayed to the viewer. The program boundary is 
sized and shaped according to parameters prescribed in the display layout. As the 
size and shape changes, the processor scales the video data for display within the 
program boundary at that instance (step 180 in Fig. 7). 

The display layout also defines how the supplemental content is illustrated 
along with the program. The display layout prescribes the size, style, location, and 
other parameters for presenting the supplemental content. For instance, the 
supplemental content might be at least partly overlaid on the video program, or 
provided as a wrapper around, or along side, the program. The supplemental 
content is displayed according to this display layout and synchronized to the 
program according to the timing information (step 182 in Fig. 7). As an example, 
the supplemental content might be a trivia game which quizzes the viewer as to 
possible outcomes of various scenes. The questions are displayed on the screen 
according to the display layout and are timed using the timing information to 
coincide with the part of the program to which the questions pertain. 
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Fig. 8b illustrates a screen 200 that is part of the interactive mode. Here, 
the video program is constrained within a program boundary 210 which is less 
than full-screen size. The program boundary 210 has been reduced in comparison 
to the full-size presentation of Fig. 8a to make room for the supplemental content. 
In this illustration, the supplemental content consists of a main menu having soft 
buttons 212-221, a highlighter or cursor icon 224, graphics bars 226, and a text 
block 228 to hold the program title. The soft buttons 212-221 present various 
control options to the viewer to invite interactive involvement with the program. 

The illustrated screen is taken from an interactive program based on a Star 
Trek episode. To provide an example of the main menu, the soft buttons 212-216 
represent actions for calling up related TV shows and movies. For instance, button 
212 relates to the original Star Trek series; button 213 concerns The Next 
Generation series; button 214 represents the Deep Space Nine series; button 215 
represents the Voyager series; and button 216 relates to the various motion 
pictures. 

Soft button 217 is an access/classified toggle which allows selection of one 
of two levels of interactivity. "Access" serves as a default mode of interactivity 
which contains the main menu functionality. The screen of Fig. 8b is an example 
of the access mode of interactivity. A voice-over narrative announces the mode 
when selected by button 217. Once in the access mode, the broadcast video stream 
is shrunk to fit within the smaller program boundary 210 to accommodate the main 
menu graphics. The alternate mode of interactivity, or "classified" mode, contains 
time-linked contextual information to provide a second tier of information. Fig. 
8c, shown below, is an example of the classified mode of interactivity. 
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Soft buttons 218-220 enable selection of different types of supplemental 
content. Button 218 relates to an encyclopedia containing numerous facts and 
information about the program. In the Star Trek program, for example, the 
encyclopedia might contain information on aliens, behind-the-scenes tidbits, cast 
and crew biographies, facts on other episodes, interviews with the actors/actresses, 
a technical manual explaining intricacies of starship operation and travel, and 
various timelines. 

Button 219 is associated with a program guide, which provides a listing of 
current Star Trek shows for a two week period. The program guide also permits 
the user to change channels to another channel carrying a Star Trek show, if 
available. 

Button 220 allows selection of entertainment content, which includes trivia 
games related to the program and a merchandise catalog. The trivia games 
segment allows viewers to compete against other regional competitors. The 
catalog lists available merchandise related to the program that can be purchased by 
the viewer. 

The tenth button 221 is a quit button which allows the viewer to leave the 
interactive mode and return to a full-screen display, as shown in Fig. 8a. 

The program and supplemental content are displayed as prescribed by the 
display layout. The digital data from the target resource, however, can 
dynamically change the display layout. When the display layout is changed (i.e., 
the "yes" branch from step 184), the viewer computing unit re-sizes and/or 
reshapes the program boundary 210 and location of the supplemental content to 
achieve the presentation envisioned by the content developer (step 186). The 
display layout can be changed for various reasons. The display layout might 
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change in response to viewer selection of a soft button, as is the case causing the 
display layout change between Figs. 8b and 8c. The display layout might also be 
altered automatically as part of the timing information. For instance, the digital 
data might invoke a graphic or text to pop up on the screen at a timely point in the 
program. Such real-time content includes, for example, trivia questions, 
interesting facts, graphical or sound effects, and so forth that relate to specific parts 
of the program. 

Fig. 8c shows the screen 200 at another instance after the viewer has 
selected soft button 220 to enter into a classified mode of interactivity. The main 
menu is still present on the screen, but more contextual information is added in a 
graphical format to represent an extension of the selected soft button 220. The 
program boundary 210 is further reduced to accommodate the additional graphical 
and textual information. 

In this illustration, the viewer has selected the entertainment button 220 to 
display the merchandise store. This layout places the video program box 210 in 
the lower right hand corner of the screen, and leaves the remaining area of the 
screen to be used for supplemental content. The supplemental content comprises a 
secondary menu having actuatable soft buttons 232-237. The cursor symbol 224 is 
movable among the various buttons and used to activate a selected button. The 
buttons correspond to various types of merchandise, such as clothes, posters, 
collectibles, memorabilia, and toys. There is also an order button 237 that permits 
the user to place an order for a particular product. A picture window 240 is 
arranged beside the secondary menu to show a picture of the merchandise 
associated with the category indicated by the cursor symbol 224. A merchandise 
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description window 242 contains a brief description of the merchandise shown in 
the picture window 240. 

The system and method for presenting interactive entertainment programs is 
advantageous as it returns the freedom of creativity to the content developer. By 
embedding the display layout instructions within the digital data provided by a 
target resource, the developer is empowered to create both the content and the 
presentation format of how the content and broadcast program are displayed to the 
viewer. The developer is free to control the location and shape of the broadcast 
program window, as well as the presentation format of the supplemental content 
relative to the program window. This facilitates a non-restrictive environment in 
which the developer can create screens with optimal appeal to the viewer. No 
longer is the developer simply developing content to be displayed within a fixed 
pane that cannot be dynamically altered relative to the program. 

The creative power is thus placed in the hands of the target resource author. 
Fig. 9 shows a method for authoring an interactive entertainment program. The 
author begins by developing the supplemental content which is to accompany a 
particular television show or movie (step 250 in Fig. 9). The author designs the 
presentation format for each scene of the program (step 252), and the timing 
requirements for synchronizing the supplemental content with the broadcast video 
stream (step 254). 

In the implementation described herein, the supplemental content is 
constructed as a hypertext document which can be rendered by a browser. To 
control how the content is to be rendered, the author encodes the presentation 
format and timing instructions as extension attributes to hyperlink codes. More 
specifically, hypermedia content utilized by the WWW is commonly written using 
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what is referred to as a "markup language." "SGML" (Standard Generalized 
Markup Language) is one such language, defined formally as "a language for 
document representation that formalizes markup and frees it of system and 
processing dependencies." SGML is a language for describing the structure of 
documents and for describing a tagging scheme to delineate that structure within 
text. For creating hypermedia content, WWW documents utilize a subset of 
SGML called "HTML" (Hypertext Markup Language). An HTML textual 
document can be thought of as plain text that contains formatting instructions in 
the form of HTML markup codes or "tags." Tags tell Web browsers how to render 
and print documents, and are also used to specify hyperlinks. 

The following is a simple example of a start tag for an HTML hyperlink: 

<A HREF=" http : //www . microsoft . com/upgrades " > 

The leading "A" in the start tag is the name of HTML element. The letter 
"A" indicates that the tag is an "anchor" tag — the type of tag that defines a 
hyperlink. The start tag contains an "attribute" indicating a target specification: 

HREF= ,r ht tp : / /www . microsoft . com/ upgrades". 

The attribute name is "HREF" and the attribute value is 
"http://www.microsoft.com/upgrades." HTML documents are organized 
according to various tags. Tags define the start and end of headings, paragraphs, 
lists, character highlighting and links, and so on. 
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The interactive entertainment system described herein supports 
conventional HTML documents as target resources. Accordingly, authors can use 
familiar HTML elements, such as Head, Body, and Character-Level elements, to 
construct a target resource. However, the inventors have also developed new types 
of tags, as well as extension attributes to existing HTML tags. The author of a 
target resource can use the new tags and extension attributes to formulate how and 
when the browser renders the supplemental content along with the continuous 
video stream. The new HTML extension attributes are given below in Table 1 . 



Table 1: HTML Extension Attributes 



HTML Tag 


Extension 
Attribute 


Function 


BODY 


background 


Specifies a URL for an image tile or MMS stream to 
cover the document background to yield a distinctive 
appearance. 


IMG 


dynsrc 


Specifies the address of a video clip or broadcast 
source to be displayed. 


FRAMESET 


usedefault 


Used to determine where the focus is located in a 
tab-based user interface within a FRAME by 
specifying the URL for the file containing the map, 
followed by a '#', followed by the name of the 
USEDEFAULT. If the argument to USEDEFAULT 
starts with a '#', the map is assumed to be in the same 
document as the USEDEFAULT tag 
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The new tags supported by the interactive entertainment system are 
provided in Table 2. 



Table 2: HTML Tags 



HTML Tag 


Function 


Permitted Attributes 


ACTION 


Controls update or 
display of sound or 
picture 


CELL — identifies obiect in a 
GALLERY 

HREF — URL to resource or obiect 
ID — label or id for this element 
TARGET — where to perform action 
such as a FRAME or 
RENDERIMAGE element 


EVENT 


Associates trigger and 
action 


TRIGGER-reference to TRIGGER 
element to be used with this event 
ACTION-reference to TRIGGER 
element to be used with this event - 
allow comment delimited list of 
actions to satisfy the need for 
multiple actions in response to a 
single trigger 
ID — name of event 


GALLERY 


Stores and coordinates 
collection of images 


SOURCE— URL to actual graphic 
ID — name of gallery 
ROWS — height of each graphic cell 
COLUMNS— width of each graphic 
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cell 

TRANSPARENCY— RGB value 
indicating background color 



LOADFONT 



Downloads and installs 
a font to be used 



HREF - URL to font resource 



RENDERIMAGE 



Displays an image from 
the GALLERY 



REF— name of GALLERY 

CELL — specifies individual cells 

containing image 



TRANSITION 



Defines screen change 
interim, fade to black, 
cut, and so on. 



TYPE -type of transition 
SPEED — how fast does the 
transition occur, SLOWEST, 
SLOW, MEDIUM, FAST, 
FASTEST 



TRIGGER 



Defines when 
something is to occur 



ID— name of this TRIGGER 
TIME — time in milliseconds since 
the last event 



Using the supplemental content and HTML tags for timing and presentation 
format, the author constructs the target resource (step 256 in Fig. 9). The target 
resource is stored in a storage medium at a host computer and a target specification 
for referencing that location is defined (step 258). The author submits the target 
specification to the authority responsible for developing the programming 
information maintained in the EPG database. The target specification is stored in 
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the appropriate data field of the EPG data structure which corresponds to the 
program to which it pertains (step 260). 

In compliance with the statute, the invention has been described in language 
more or less specific as to structure and method features. It is to be understood, 
however, that the invention is not limited to the specific features described, since 
the means herein disclosed comprise exemplary forms of putting the invention into 
effect. The invention is, therefore, claimed in any of its forms or modifications 
within the proper scope of the appended claims appropriately interpreted in 
accordance with the doctrine of equivalents and other applicable judicial doctrines. 
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CLAIMS 




A viewer computing unit for receiving and displaying continuous 



video content programs, comprising: 
a memory; 

a processor programmed to determine whether the video content programs 
are interactive; 

a tuner to tune to channels carrying the video content programs; and 
an interactive support module stored in the memory, the interactive support 
module being dynamically loadable for execution on the processor when the tuner 
is tuned to a channel carrying a video content program that is interactive. 

w 

^J£S A viewer computing unit as recited in claim 1, wherein the interactive 
support module comprises a hyperlink browser. 



4. A viewer computing unit as recited in claim 1, and further 
comprising: 

an electronic programming guide (EPG) stored in the memory and 
executable on the processor to organize programming information, the EPG 
associating a target specification to a target resource with a video content program; 
and 

the interactive support module activating the target resource when the tuner 
is tuned to the video content program. 



i 3 - 




support module comprises an Internet browser. 
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5. A viewer computing unit as recited in claim 4, wherein the target 
resource contains supplemental content which is displayed concurrently^with the 
video content program to provide viewer interactivity with the video content 
program and display layout instructions prescribing how the supplemental content 
and the video content program are to appear in relation to one another when 
displayed, the processor being responsive to the layout instructions obtained from 
the target resource to display the supplemental content concurrently with the video 
content program. 

6. A viewer computing unit as recited in claim 4, further comprising: 

a receiver coupled to the processor to receive both the video content 
program and data supplied from the target resource. 

7. A viewer computing unit as recited in claim 4, further comprising: 

a first receiver coupled to the processor to receive the video content 
program; and 

a second receiver coupled to the processor to receive data supplied from the 
target resource. 
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( ' "^8. In an interactive entertainment device having an ability to receive and 
display television signals, the interactive entertainment device supporting a 
displayable user interface (UI), a method for operating the interactive 
entertainment device comprising the following step of displaying a hyperlink to a 
target resource in the UI to enable a viewer to activate the target resource directly 
from the UI by activating the hyperlink. 

9. A method for enhancing a continuous video content program with 
supplemental hyperlink content to provide viewer interactivity with the video 
content program, comprising the following steps: 

configuring digital data which defines a display layout prescribing how the 
supplemental hyperlink content and the video content program are to appear in 
relation to one another when displayed; 

transmitting the digital data and the video content program to a viewer 
computing unit; and 

displaying the supplemental hyperlink content and the video content 
program according to the display layout. 

10. A method as recited in claim 9, further comprising the following 

steps: 

configuring the data to define multiple different display layouts that are 
selectively displayed to the viewer depending upon the viewer's selections of 
possible choices presented in the supplemental hyperlink content; and 

dynamically changing the display layouts of the supplemental hyperlink 
content and the video content program in response to said viewer's selections. 
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11. A method as recited in claim 9, wherein the transmitting step 
comprises the step of transmitting the digital data along with the videp content 
program as the same signal. 

12. A method as recited in claim 11, further comprising the following 

steps: 

receiving said signal containing the digital data and the video content 
program at the viewer computing unit; and 

separating the digital data from the video content program at the viewer 
computing unit. 

13. A method as recited in claim 9, wherein the transmitting step 
comprises the step of transmitting the digital data along with the video content 
program as two separate signals. 

14. A method as recited in claim 13, further comprising the following 

steps: 

receiving a first signal containing the digital data using a first receiver at the 
viewer computing unit; and 

receiving a second signal containing the video content program using a 
second receiver at the viewer computing unit. 



Lee & Hayes, PLLC 



3 3 MSU 39279 0329961414 C. \LHShared\MStt089tt*\MSI-089US.pat app doc 



1 

2 

4 
5 
6 
7 
8 

A 

if 

9 
10 
11 

13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



15. A method as recited in claim 9, wherein the transmitting step 
comprises the following steps: 

transmitting the digital data as a first signal from a first source; and^ 
transmitting the video content program as a second signal from a second 
source that is different than the first source. 

16. A method as recited in claim 9, wherein the configuring steps 
comprises the step of creating an HTML document having HTML extension 
attributes that assist in defining the display layout. 

17. A method as recited in claim 16, further comprising the step of 
creating the HTML document using at least one extension attribute selected from a 
group comprising: a background extension attribute which specifies how a 
background is to appear; an image source extension attribute which specifies an 
address of a video source to be displayed; and a focus extension attribute to specify 
where a focus indicia is located in the display. 

18. A method as recited in claim 9, wherein the configuring steps 
comprises the step of creating an HTML document having one or more HTML 
tags that assist in defining the display layout. 
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19. A method as recited in claim 18, further comprising the step of 
creating the HTML document using at least one tag selected from a group of tags 
comprising: a tag to control update or display of sound or pictures; a tag to store 
and coordinate collections of images; a tag to control font styles; a tag to retrieve 
and display one of the images; and a tag to describe transition from one screen 
display to another. 

^20. A method for presenting an interactive program, comprising the 
following steps: 

receiving a program as a continuous stream of video data; 

receiving digital data for supporting interactive functionality in relation to 
the program; 

displaying the program within a program boundary on a visual display 

screen; 

presenting supplemental content from the digital data in a presentation 
format on the visual display screen which enables the interactive functionality; and 

dynamically controlling location and shape of the program boundary and 
the presentation format of the supplemental content relative to the program 
boundary on the visual display screen. 

A method as recited in claim 20, further comprising the step of 
overlaying the supplemental content at least partly on the program displayed 
within the program boundary. 
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t , v ^22. A method as recited in claim 20, further comprising the step of 
presenting the supplemental content outside of the program boundary. 

\ : [ 23. A method as recited in claim 20, further comprising the step of 
synchronizing presentation of the supplemental content to corresponding points in 
the program. 

' 24. A computer programmed to perform the steps recited in claim 20. 

25. A computer-implemented method for activating interactive 
supplemental content for a video content program upon tuning to a channel 
carrying the program, comprising the following steps: 

determining if the program is interactive compatible, where interactive 
compatible programs are associated with target resources containing data which 
support interactive functionality in conjunction with the associated programs, the 
target resources being located by corresponding target specifications; and 
Su in an event that the program is interactive compatible, retrieving a target 
specification associated with the program and launching code to activate the target 
resource in support of interactive functionality for the associated program. 

26. A computer-implemented method as recited in claim 25, wherein the 
target specifications are correlated with associated programs in a program listing, 
and further comprising the following steps: 

checking the program listing to ascertain whether the program is interactive 
compatible; and 
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determining that the program is interactive compatible by presence of a 
target specification being associated with the program in the program listing. 

27. A computer-implemented method as recited in claim 25, wherein 
said determining step comprises the step of checking a channel separate from said 
program channel for presence of the supplemental content in conjunction with the 
program being received on said program channel. 

28. A computer-implemented method as recited in claim 25, further 
comprising the step of displaying an icon to visually inform the viewer that the 
program is interactive compatible. 

29. A computer-implemented method as recited in claim 28, further 
comprising the step of displaying the interactive supplement content in response to 
the viewer activating the icon. 

30. A computer-implemented method as recited in claim 25, further 
comprising the step of automatically displaying the interactive supplement content 
together with the interactive compatible program. 

^31. A computer-implemented method as recited in claim 25, further 
comprising the step of loading a hyperlink browser to activate the target resource. 

32. A computer programmed to perform the steps recited in claim 25. 
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33. A method for creating a data structure in a storage medium that is 
used to organize programming information, comprising the following steps: 

forming data fields in a storage medium to hold programming information 
pertaining to video content programs, some of the data fields holding text-based 
data; and 

adding a target specification which references a target resource containing 
data that supports interactive functionality with respect to various ones of the video 
content programs by at least one of (1) forming a separate data field to hold the 
target specification for an associated video content program, or (2) embedding the 
target specification within the text-based data held in a data field. 

34. A computer programmed to perform the steps recited in claim 33. 

35. A storage medium having a data structure created according to the 
steps recited in claim 33. 

36. A method for authoring an interactive entertainment program, 
comprising the following steps: 

constructing digital data to support interactive functionality with a video 
content program, the digital data being configured to permit a viewer to 
interactively control display of supplemental content along with the video content 
program; 

defining a display layout of how the supplemental content and the video 
content program are displayed; and 



Lee & Hayes, PLLC 



3 8 MS# 39279 0329961414 C:\IMSharedMS I\0S9uMSI-O89US.pat,app doc 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
p23 
24 
25 



encoding the digital data with instructions to dynamically change the 
display layout of the supplemental content and the video content program. 

37. A method as recited in claim 36, further comprising the step of 
encoding the digital data with instructions to dynamically change the display 
layout in response to viewer control. 

38. A method as recited in claim 36, further comprising the following 

steps: 

developing timing information to synchronize presentation of the 
supplemental content in conjunction with the video content program; and 

encoding the digital data with instructions to alter the display layout of the 
supplemental content and the video content program in response to the timing 
information. 

39. A method as recited in claim 36, further comprising the following 
step of storing the digital data with instructions as a target resource in a storage 
medium. 

40. A target resource stored in a storage medium which is constructed 
according to the steps recited in claim 39. 

41. A computer programmed to perform the steps recited in claim 36. 
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42. A computer-implemented method comprising the following steps: 
tuning to a channel; 

determining if a video content program being carried on the channel is 
interactive compatible as indicated by presence of a target specification provided 
in association with the video content program; 

in an event that the program is interactive compatible, retrieving the target 
specification associated with the video content program on the channel; 

launching a browser to activate a target resource located by the target 
specification, the target resource containing digital data which supports interactive 
functionality in conjunction with the associated video content program, the digital 
data defining supplemental content to enable viewer interactivity with the video 
content program and a display layout prescribing how the supplemental content 
and the video content program are to appear in relation to one another when 
displayed; 

receiving the video content program over the channel; 

receiving the digital data from the target resource; and 

displaying the video content program and the supplemental content 
according to the display layout prescribed in the digital data received from the 
target resource. 

43. A computer-implemented method as recited in claim 42, further 
comprising the following steps: 

correlating the target specifications with associated programs in an 
electronic program guide (EPG); and 
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checking the EPG to ascertain whether the program is interactive 
compatible; and 

determining that the program is interactive compatible by presence of a 
target specification in the EPG which is related to the program. 

44. A computer-implemented method as recited in claim 42, further 
comprising the following steps: 

displaying the video content program within a program boundary on a 
visual display screen; 

presenting the supplemental content in a presentation format on the visual 
display screen; and 

controlling location and shape of the program boundary and the 
presentation format of the supplemental content relative to the program boundary 
according to the display layout received from the target resource. 

45. A computer-implemented method as recited in claim 44, further 
comprising the step of synchronizing presentation of the supplemental content to 
corresponding points in the video content program. 

46. A computer-implemented method as recited in claim 42 ; further 
comprising the following steps: 

receiving the video content program from a first source; and 
receiving the digital data from the target resource at a second source 
different than the first source. 
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47. A computer-implemented method as recited in claim 42, wherein the 
digital data at the target resource further defines timing information to synchronize 
presentation of the supplemental content with the video content program, and 
comprising the step of displaying the supplemental content at prescribed times 
during the video content according to the timing information received from the 
target resource. 

48. A computer-implemented method as recited in claim 42, further 
comprising the step of displaying an icon to visually inform the viewer that the 
program is interactive compatible. 

49. A computer-implemented method as recited in claim 48, further 
comprising the step of displaying the supplement content in response to the viewer 
activating the icon. 

50. A computer-implemented method as recited in claim 42, further 
comprising the step of automatically displaying the supplement content together 
with the video content program. 

51. A computer-implemented method as recited in claim 42, further 
comprising the step of overlaying the supplemental content at least partly on the 
video content program. 
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52. A computer-implemented method as recited in claim 42, further 
comprising the step of presenting the supplemental content in an area surrounding 
the video content program. 

53. A computer programmed to perform the steps recited in claim 42. 

54. A computer-readable storage medium containing a target resource, 
the target resource comprising: 

supplemental content for rendering to a viewer to supplement viewing of a 
continuous, non-interactive video stream; 

one or more elements prescribing how the supplemental content is to be 
rendered along with, and relative to, the video stream. 

55. A computer-readable storage medium as recited in claim 54 wherein 
;the target resource comprises an HTML document and the elements comprise 
HTML tags and/or extension attributes for HTML tags. 
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ABSTRACT 

An interactive entertainment system enables presentation of supplemental 
interactive content along side traditional broadcast video programs, such as 
television shows and movies. The programs are broadcast in a conventional 
manner. The supplemental content is supplied as part of the same program signal 
over the broadcast network, or separately over another distribution network. A 
viewer computing unit is located at the viewer's home to present the program and 
supplemental content to a viewer. When the viewer tunes to a particular channel, 
the viewer computing unit consults an electronic programming guide (EPG) to 
determine if the present program carried on the channel is interactive. If it is, the 
viewer computing unit launches a browser. The browser uses a target specification 
stored in the EPG to activate a target resource containing the supplemental content 
for enhancing the broadcast program. The target resource contains display layout 
instructions prescribing how the supplemental content and the video content 
program are to appear in relation to one another when displayed. When the data 
from the target resource is downloaded, the viewer computing unit is responsive to 
the layout instructions obtained from the target resource to display the 
supplemental content concurrently with the video content program. Embedding 
the layout instructions in the supplemental content advantageously places control 
of the presentation to the content developers. The developers are free to arrange 
the data and video in any manner they choose. 
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From: Lewis Lee 
Sent: Thursday, February 22, 1996 5:34 PM 
To: Dan Hayes 

Subject: Patent Summary: "Enhanced Broadcasts" MS# 39279 / MS1-089US 



Inventorship : Valerie Bronson, Dan Shoff, Joe Matthews, Frank Lawler 



Invention Summary : This invention concerns a technique for placing graphics and other web stuff 
over or around a continuous video stream to enhance a broadcast. The system separates the 
program stream from the Web content so that they are not inseparably intertwined (as are the 
prior art). This ailows'the developer to design the overlay or enhancements in relation to the 
video picture however they wish. For instance, this approach enables the developer to move the 
web content around on the screen and to reposition or resize the video picture. 

The viewer can activate the enhanced broadcast manually, by being told in the W Guide that a 
particular program is interactive. 

An alternative approach is to have the operating system intelligently determine whether the 
program is interactive. This is accomplished by placing a web extension into a separate field of 
the EPG. When the viewer tunes to a particular channel, the navigator checks the EPG to see 
this field contains an extension, indicating that the program is interactive. If so, the web browser 
can be launched automatically, or a visual cue (akin to the stereo icon) can be depicted on the TV 
to inform the viewer that the program is interactive. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Application Serial No. 

Filing Date 

Inventor 

Applicant 

Attorney's Docket No. 



08/623,610 

March 29, 1996 

Shoffet al. 

Microsoft Corporation 
MS1-089US 



Title: Interactive Entertainment System for Presenting Supplemental Interactive 
Content Together with Continuous Video Programs 

DECLARATION FOR PATENT APPLICATION 

AND POWER OF ATTORNEY 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to 
my name, 

I believe I am the original, first and sole inventor (if only one name is listed 
below) or an original, first and joint inventor (if plural names are listed below) of the 
subject matter which is claimed and for which a patent is sought on the invention 
entitled "Interactive Entertainment System for Presenting Supplemental Interactive 
Content Together with Continuous Video Programs/' the specification of which is 
attached hereto. 

I have reviewed and understand the content of the above-identified 
specification, including the claims. 

I acknowledge the duty to disclose information which is material to the 
examination of this application in accordance with Title 37, Code of Federal 
Regulations, § 1.56(a). 

PRIOR FOREIGN APPLICATIONS: no applications for foreign patents or 
inventor's certificates have been filed prior to the date of execution of this 
declaration. 
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Power of Attorney 

I appoint the following attorneys to prosecute this application and transact all 
future business in the Patent and Trademark Office connected with this application: 
Lewis C. Lee, Reg. No. 34,656, and Daniel L, Hayes, Reg. No. 34,618. 

Send correspondence to: LEE & HAYES, PLLC, W. 1818 Francis #160, 
Spokane, Washington, 99205. Direct telephone calls to: Lewis C. Lee (509) 324- 
9256. 



All statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that 
these statements were made with the knowledge that willful false statements and the 
like so made are punishable by fine or imprisonment, or both, under Section 1001 of 
Title 18 of the United States Code and that such willful false statement may 
jeopardize the validity of the application or any patent issued therefrom. 
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381 Crystal Creek Cir. 
Issaquah, WA 98027 
USA 
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Full name of inventor: 
Inventor's Signature: _ 
Date: Jy^ /o flU 
Residence: 

Citizenship: 

Full name of inventor: 
Inventor's Signature: _ 
Date: C/f6 fid 
Residence: 

Citizenship: 



Full name of inventor: Frank Lawler 




Residence: 4459 Freemont Ave. N. #302 

Seattle, WA 98103 
Citizenship : Canadian 



Valerie L. Bronson 



203 Bellevue Way NE #269 
Bellevue, WA 98004 
USA 



Joseph H. Matthews, III 



1907 232nd PI. N.E. 
Redmond, WA 98053 
USA 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Priority Serial No 

Priority Filing Date 

Inventorship 

Priority Group Art Unit 

Priority Examiner 

Applicant 

Attorney's Docket No. . 



08/623,610 

3/29/96 

Shoffetal. 
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John W, Miller 

Microsoft Corporation 



MS1-089USC1 



Title: Interactive Entertainment System For Presenting Supplemental Interactive 
Content Together With Continuous Video Programs 



As a below named inventor, I hereby declare that: 

I am the inventor of the subject matter which is claimed and for which a 
patent is sought on the invention entitled "Interactive Entertainment System for 
Presenting Supplemental Interactive Content Together with Continuous Video 
Programs", as identified above. 

I conceived of the invention in the United States prior to the March 8, 1996 
priority date of U.S. Patent No. 5,778,181 to Hidary et al. (hereinafter, "Hidary 
patent"). Attached to this declaration are excerpts from a patent disclosure summary 
prepared by our attorney Lewis C. Lee following a disclosure meeting. The 
summary briefly describes aspects of the invention. The disclosure meeting was 
held and the disclosure summary was drafted prior to the priority date of the Hidary 
patent. Diligent effort was made between conception of the invention, as evidenced 
by this disclosure summary, and the filing of a patent application on March 29, 1996. 

All statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that 
these statements were made with the knowledge that willful false statements and the 
like so made are punishable by fine or imprisonment, or both, under Section 1001 of 
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Title 18 of the United States Code and that such willful false statement may 
jeopardize the validity of the application or any patent issued therefrom. 



Full name of inventor: 

Inventor's Signature 

Residence: 

Citizenship: 

Post Office Address: 



Daniel J 




Issaquah, W. 
USA 

381 Crystal Creek Cir 
Issaquah,WA 98027 



Date: 



Full name of inventor: 

Inventor's Signature 

Residence: 

Citizenship: 

Post Office Address: 



Valerie L. Bronson 

Bellevue, WA 
USA 

203 Bellevue Way NE #269 
Bellevue, WA 98004 



Date: &/j?x/q4 



Full name of inventor: 

Inventor's Signature 

Residence: 

Citizenship: 

Post Office Address: 



Joseph H. Matthews, III 
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^ Date: L l /tf/ff 
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Cedmond, WA 
USA 

1907 232nd PI. N.E. 
Redmond, WA 98053 
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Full name of inventor: Frank A. Lawler 

Inventor's Signature „ Date: ^fa/?/ 

Residence: Seattle, WA 

Citizenship: Canada 
Post Office Address: 
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